Tom's Insights - Week of 7/9/01

Week of July 9th, 2001

Last Updated: July 16, 2001 18:27

Monday Tuesday Wednesday Thursday Friday Saturday Sunday Most Recent
Last Week Next Week Insights Index Daynotes.com Email: tom@syroidmanor.com

Monday July 9, 2001

As you've probably already surmised, my weekend was a hectic one.

We arrived home late Saturday with two ice cream buckets of sasktoons to a house that had been closed up (in 33 degree temps) for the better part of 9 hours. Needless to say, we spent the evening on the porch waiting until it was "safe" to go inside -- which was about 11:30 pm.

Yesterday I sequestered myself in my office (with a box fan set up in the window and another at the door) and set to work bringing Hydras back to an operational state. Like everything in the world of bits and bytes, it took longer than I anticipated -- I actually wanted to finish things up yesterday and be online today. No joy, but I did learn lots and got a fair amount of dirt under my finger nails regarding AIX 5.1.

I began the day by trying to get the Apache Toolbox to function under AIX. Unfortunately, I did not succeed. The script was written to function under Linux/Bash and the Korn shell and the AIX filesystem present a unique enough environment that things fall down in several spots. Too bad, because it is truly a unique and very cool product. I late tried it out on Phoenix (just running the script; not the make install part) and it worked flawlessly. I sent the author the error outputs I rec'd; hopefully he can adapt things enough to one day run on AIX. It sure does make setting up a custom web server as simple as "point and click". Recommend if you run one of the systems it has been tested under.

Next I pulled down the source for Apache 1.3.20 and tried to compile it. Again, no joy. The configuration script was dying when it failed to find "libm". Mmm. Never heard of it, but a quick search showed a libm.a buried several layers down in an obscure part of the /usr filesystem. So I tried to symlink it up into /usr/lib, but I botched that up by reversing the target/destination and effectively overwriting the file with the link. Bah and Piffle. With some help from Ken Coar this morning I've finally discovered that Apache is looking for the AIX math library, which is not installed by default for whatever reason. I found what I needed on CD1, and installed libm without incident to /usr/lib (I have no idea what the "other" libm file does). I tried to compile 1.3.20 again this morning, and it finds the library, but it's now complaining that it's not a COFF file. WTF is a COFF file?? Sigh.

I finally found a pre-built TAR of 1.3.20 for AIX over on the ULCA site and installed that for now. I still want to compile my own custom installation, but first I need to find out what a COFF file is and (hopefully) find a workaround.

I also got Samba installed and dancing yesterday. I decided to simply pull the 2.2.0 BFF package off the Bull download site; 2.2.1 should be finalized in a week or two and when it is I'll build from source then.

I rebuilt all my RAID arrays last night (not required, but I wanted to move some file systems around), and this morning I finally finished rsync'ing all my data files back to Hydras that are sitting on Phoenix. Now all that remains is to tweak up some configuration files, build a default profile template, and add users back into the fray. All relatively mechanical, but still time consuming. Hopefully I'll get things finished up tonight so I can run the box in its finished configuration for a day or two just to ensure nothing breaks and the error logs remain clean.

-oOo-

Sometime I have no other option but to laugh at some of the stuff that lands in my Inbox. This was address specifically to tom@syroidmanor.com...

**dear sir tom hanks**
with regards and greetings
by the occasion july 9 **happy birthday to you**
my name is ali reza mesgarnia,i am sending this letter from tehran for you.
you have the highest personality,i like you.
from almighty god i wish health for you and your respectable family.
**dear sir tom hanks**,
i do apologize for:four years ago  i am sufferring from lumbago and column 
failure.
i pray you please favour me with your help and assistance.i kiss your hands  
from far distance.please excuse me.
thanks and regards *the best wishes for you*
_ faithfully yours
alireza  mesgarnia
july 8,2001
my mailing address:home no.27,3nd.floor,
hassan sokhanvar alley,
after sepah cross_roads,
valiasr st.,
tehran_iran
postal code:11376  phone:(0098)(21)_6463307

Yup. OK. Whatever. Have a great evening... and *the best wishes for you too*

[Top]

Tuesday July 10, 2001

[Top]

Wednesday July 11, 2001

<VBS>... I put up a post yesterday. Really. As a matter of fact, I did it twice; the first one vanished in one of several power outages we endured last night. I honestly dunno what happened. It was RIGHT THERE (↑) last time I looked.

Bother. If anyone has my post of yesterday in their cache, please send it home. Thanks.

To recap: Yesterday I sorted out my EGD (entropy gathering daemon) so it would work with OpenSSH. It was simply a matter of changing one of the configuration/compile-time option. One of those very non-intuitive things. And Samba 2.2.1 continue to run like a top. The only remaining piece I had to contend with was a profile template, re-creating users, and getting Apache to compile. I was successful on two out of three.

I had one other stumbling block earlier in the day, however. The power outages last night were a good test to ensure everything returned to proper form after a power failure (the UPS has only enough batter to keep Hydras alive for about 15 minutes; after that it dies, but it's programmed to automatically restart when the power comes on). After numerous attempts, I got the entropy gathering perl script to start from inittab, but OpenSSH and Samba would not initialize. I keep all non-OS startup commands in /usr/local/ect/rc.local, which is initialized from inittab. But for some reason rc.local was not happening, and I was miffed and stumped.

The short story is that (a) I forgot to make rc.local executable when I created it, and (b) I had to move the command to start EGD higher in the startup sequence to give it a chance to build a random number pool. Turns out I was trying to start OpenSSH before the random pool had a chance to fully build, which killed the script. And because I had SSHD starting before Samba, when the script died, it died. The fix was an easy one; just start EGD higher in /etc/inittab.

Today was consumed with research, a little editing, translating a plain text tutorial into XML, and building up my default "make user" scripts under AIX. I ended up getting my own personal profile tweaked up fairly well over the years, but I never took the time to polish the edges and revamp the mkuser.sys script so that the three configuration files I created (.profile = sets the prompt, user environment, etc; .kshrc = sets aliases; .kshexit = cleans up when the user logs out) would be copy to ~/home when a new user was created. Today I did just that.

I also reverted to Apache 1.3.19 (installed from a bff package) because I could not for the life of me get 1.3.20 to compile under AIX. The situation even has Ken Coar (one of the lead developers of Apache) stumped. Oh well. The difference between 1.3.19 and 1.3.20 are primarily fixes to make Apache run more civilized under NT.

I should be ready to move all services back to Hydras late Saturday night. I'm getting one very annoying message in the error log regarding a duplicate IP on the net (I've double-checked; all machines on my network have unique IP's), but other than that all is well and good. If you have a shell account on Hydras that you're using for file storage -- don't until I give the nod this weekend. Web updates are fine and to be expected, but I spent several hours resetting permissions and generally organizing the new "user" filesystem, so I won't be resync'ing the /home directory on Phoenix again apart from the web stores. When I move the web stores to Hydras -- probably late Saturday -- I'll then clean up that branch of the tree.

Something to note if you're using a script to rsync or scp files to Hydras on a daily basis: The new "user" filesystem is now /export/home/username instead of /home/username. I made the change because the process of removing the initial /home partition is difficult and fraught with potentially Ungood consequences, and because eventually I'll be moving to either LDAP or NIS+ as a central directory system and /export/home is the recommended structure.

[Top]

Thursday July 12, 2001

A friend I haven't seen for 20 years was passing through Saskatoon and dropped in to say HI. Great fun talking about "old times"...

So my post today will be short -- I need some help from any compiler gurus out there.

I'm trying to build BIND 9.1.3 on Hydras (AIX 5.1, GNU Make 3.74, GNU GCC 2.95.3). I get the following error:

> ld: 0711-781 ERROR: TOC overflow. TOC size: 67872       Maximum size:
> 65536 collect2: ld returned 12 exit status
> make: 1254-004 The error code from the last command is 1.
>
>
> Stop.
> make: 1254-004 The error code from the last command is 1.
>
>
> Stop.
> make: 1254-004 The error code from the last command is 1.
>
>
> Stop.
> [root@hydras]:/usr/local/src/bind-9.1.3 #

Anyone know what a TOC overflow is? I've never experienced it myself. Brian did a Google search and suggested I try the option --with-bigtoc, but that didn't do squat.

I'm baffled. Help.

[Top]

Friday July 13, 2001

Not worth a poop, but thanks for asking...

The TOC error I stumbled upon yesterday is still giving me heartburn. Several people -- all of whom I consider to be the brightest minds in the business -- responded with ideas, but I've yet to make any of them work. Or perhaps it is my implementation. I'm not sure. Many thanks to Moshe, Brian, Roland, and Peter Thomas for your input and ideas.

Roland wrote the most detailed summary:

TOC is the Table Of Contents generated for every executable file.
-mfull-toc is the default, which causes gcc to GNU CC to allocate at least
one TOC entry for each unique non-automatic variable reference in your  
program. It'll also place floating-point constants in the TOC.
However, only 16,384 entries are available in the TOC.  

You may wish to try the following options:

 -mno-fp-in-toc -mno-sum-in-toc

If that doesn't work, try this:

 -mminimal-toc

If you minimize the TOC values, your code will be larger, and run a bit
more slowly, because then sums of addresses and constants must be
calculated at runtime, rather than being precalculated and stored in the
TOC.  If you specify -bigtoc, the inverse is true. 

Note that this isn't, strictly speaking, a compilation error, but rather a
linking error (from ld).

Moshe also said to try -mminimal-toc, as did Brian and Peter (all backed up with URL's and research). So far, however, I'm having no joy. Come to think of it, I'm beginning to think AIX is cursed when it comes to building BIND. I've yet to do so without some sort of strange and unfixable error. Bah.

I've tried:

passing the mminimal-toc option to ./configure as a --with argument (./configure --with-mminimal-toc). Nope. Falls down with the same TOC overflow error.
passing -mminimal-toc to the make comand. Nope.
editing Makefile to include the command CXXFLAGS=-mminimal-toc (see http://mail.gnu.org/pipermail/bug-ddd/2000-October/000026.html )
plus another 3 or 4 other suggested options.

Humbug.

[Top]

Saturday July 14, 2001

Yeeshhh. Very Warmy here tonight. Hot and humid. I'm not complaining, mind you -- just stating a fact. Having lived in -40 a time or two, one learns not to piss off Mother Nature by whining about the heat.

Progress has been made on the BIND front. Sorta-kinda. With a tip of the hat to Mr. Bilbrey for his research, the commands to build BIND on Hydras under AIX 5.1 are:

make clean
./configure
export EXT_CFLAGS=-mminimal-toc
make
make install

Which builds BIND 9.1.3 without errors. Now if I could just get it to run... [sigh]

/usr/local/sbin/bind -v produces:

BIND 9.1.3

(so we know the executable is working). /usr/local/sbin/named -c /etc/named.conf executes without error, but no process is actually started. Which tells me there is an error somewhere in the config file (that's what both Apache and Samba do if there's an error in the config file; they look like they're starting but then immediately die). So. To test the configuration file, I fired up the BIND version installed with the AIX 5.1 (8.2.2). It wants the config file to be called named.boot, so we cp /etc/named.conf /etc/named.boot. Then fire up the program (startsrc -s named), and viola, off it goes. So as far as NAMED 8.2.2 is concerned, the config file is good. And I'm using the same config and zone files that have been running under BIND 9.1.0 on Donovan for six months now. Mmm. Dunno. Again.

-oOo-

I ordered a new notebook yesterday. A Dell Inspirion 8000. I had a lot of people write me and recommend both Compaq and IBM notebooks. I had an IBM ThinkPad once, and it does indeed have the best keypad in the business. And contrary to some folk, I kinda like the little red stick thiugy in the middle of the layout -- more so than a touchpad. But dollar for dollar, option for option, no one could beat Dell's package. And given that my employer is [indirectly] buying it for me, price is definitely a factor. At the same time, I wanted a machine that would last me three years or so.

For $2700 US (and a bit of change), I got:

PIII-1GHz processor
256 MB RAM (one DIMM; which leaves one open slot)
15" UltraXGA screen
20GB HDD
nVidia 32MB chipset (<-- big sway for me; I LIKE nVidia chipsets and Dell is currently the only notebook supplier using them)
24X CDROM
CDRW/DVD combo
Built-in 56K modem and 10/100 NIC
Windows 2000 Pro
Office XP SB edition
Leather case
USB Intellimouse
Extra battery

I'm pretty excited. I've never owned a brand new notebook, and I haven't bought a new machine for almost two years. Amazing when you think about it. The system weighs around 6.5 pounds and can be tucked in a case and carried around on my shoulder. And it will be the fastest system yet at Syroid Manor. Also amazing is the fact the same system nudged close to $5000 just one year ago.

[Top]

Sunday July 15, 2001

23:00 hrs... Late post -- we're just in the door.

Helped our neighbors move today. The temperature at 9am was close to 30 degrees C. By 2pm it was 34. I was a very long, very hot day, but we had lots of hands, so it went quickly. We had everything moved by 3pm and most of the furniture assemble and in place by 6. Then it was Miller time, accompanied by gourmet hamburgers on the BBQ.

All in all, a successful day.

And while I was sweating in the heat, Brian was sweating over BIND on Hydras -- remotely, from the comfort of his air conditioned California apartment. Geesh I love and Internet and SSH <g>.

First trick was to get BIND to produce an event log. AIX has extensive logging capabilities, but everything must be configured manually. So Brian added an entry to syslog.conf, restarted the logging daemon, killed the BIND version I had running (8.2.2), and started BIND 9.1.3. The problem was immediately evident. BIND was trying to write its PID to /var/run, and /var/run did not exist.

Despite all his hard work Brian was getting some odd responses from nslookup last night, so we're not there yet, but we're definitely getting closer.

Hope everyone had a great weekend...

[Top]